<!doctype html>
<html lang="en">
<head>
    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <meta name="description" content="coinfaces">

        <title>Visual Studio Code inotify 资源耗尽的问题</title>

    <link rel="stylesheet" href="../theme/css/style.css">
    <link rel="stylesheet" href="../theme/css/pygments.css">



</head>

<body>
    <div class="content">
<h1>Visual Studio Code inotify 资源耗尽的问题</h1>    <p>
        under
            <a href="../tags/inotify/">inotify</a>
            <a href="../tags/vscode/">vscode</a>
    </p>
    <p>
        in <a href="../categories/uncategorized/">Uncategorized</a>
    </p>
    <p>Published: 2019-04-04</p>


    <!-- wp:paragraph -->

<p>开着 vscode 和 quasar dev -m spa，经常出现 vscode 里的代码更新了，quasar
dev 却没有重新编译代码，两者是不是还报 ENOSPC 的错误。用 tail -f
测试一下：</p>
<!-- /wp:paragraph -->

<!-- wp:code -->

<div class="highlight"><pre><span></span><code>$ tail -f ~/.bashrc
...
tail: inotify resources exhausted
tail: inotify cannot be used, reverting to polling
</code></pre></div>


<!-- /wp:code -->

<!-- wp:paragraph -->

<p>果然是（当前用户的）inotify 监控资源耗尽了。</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->

<p>现在 vscode 的 files.watcherExclude 设置如下：</p>
<!-- /wp:paragraph -->

<!-- wp:code -->

<div class="highlight"><pre><span></span><code><span class="err">{</span>
<span class="err">  &quot;files.exclude&quot;: {</span>
<span class="err">    &quot;.quasar&quot;: true,</span>
<span class="err">    &quot;**/node_modules&quot;: true,</span>
<span class="err">    &quot;dist&quot;: true,</span>
<span class="err">    &quot;src-cordova&quot;: true</span>
<span class="err">  },  </span>
<span class="err">  &quot;files.watcherExclude&quot;: {</span>
<span class="err">    &quot;**/.git/objects/**&quot;: true,</span>
<span class="err">    &quot;**/.git/subtree-cache/**&quot;: true,</span>
<span class="err">    &quot;**/node_modules/*/**&quot;: true,</span>
<span class="err">    &quot;**/src-cordova/**&quot;: true</span>
<span class="err">  }</span>
<span class="err">}</span>
</code></pre></div>


<!-- /wp:code -->

<!-- wp:paragraph -->

<p>但还是时不时出现这样的问题，用下面的命令可以看到 inotify
监控数量的上限是8192：</p>
<!-- /wp:paragraph -->

<!-- wp:code -->

<div class="highlight"><pre><span></span><code><span class="err">cat /proc/sys/fs/inotify/max_user_watches</span>
</code></pre></div>


<!-- /wp:code -->

<!-- wp:paragraph -->

<p>用 lsof 可以找出哪些继承使用了 inotify
监控，但不能具体列出每个进程使用了多少：</p>
<!-- /wp:paragraph -->

<!-- wp:code -->

<div class="highlight"><pre><span></span><code><span class="err">lsof | grep inotify</span>
</code></pre></div>


<!-- /wp:code -->

<!-- wp:paragraph -->

<p>更精确的信息需要到 /proc 目录中去寻找，下面的命令列出了所有进程使用的
inotify 监控的文件描述符：</p>
<!-- /wp:paragraph -->

<!-- wp:code -->

<div class="highlight"><pre><span></span><code><span class="err">find /proc/*/fd/* -type l -lname anon_inode:inotify 2&gt;/dev/null</span>
</code></pre></div>


<!-- /wp:code -->

<!-- wp:paragraph -->

<p>输出如下信息：</p>
<!-- /wp:paragraph -->

<!-- wp:code -->

<div class="highlight"><pre><span></span><code><span class="err">/proc/2335/fd/37</span>
<span class="err">/proc/23924/fd/49</span>
<span class="err">/proc/23924/fd/70</span>
</code></pre></div>


<!-- /wp:code -->

<!-- wp:paragraph -->

<p>可以看到，有的进程使用了1个文件描述符，每个进程使用了多个文件描述符。每个文件描述符对应多少个监控数，需要在
/proc/&lt;pid&gt;/fdinfo/&lt;fd&gt; 中查看：</p>
<!-- /wp:paragraph -->

<!-- wp:code -->

<div class="highlight"><pre><span></span><code><span class="err">cat /proc/2335/fdinfo/37</span>
</code></pre></div>


<!-- /wp:code -->

<!-- wp:paragraph -->

<p>可以看到这样的内容，每个 inotify 开头的行对应着1个监控数。</p>
<!-- /wp:paragraph -->

<!-- wp:code -->

<div class="highlight"><pre><span></span><code><span class="n">pos</span><span class="o">:</span>   <span class="mi">0</span>
<span class="n">flags</span><span class="o">:</span>  <span class="mi">02004000</span>
<span class="n">mnt_id</span><span class="o">:</span> <span class="mi">13</span>
<span class="n">inotify</span> <span class="n">wd</span><span class="o">:</span><span class="n">d</span> <span class="n">ino</span><span class="o">:</span><span class="mi">823</span><span class="n">ba</span> <span class="n">sdev</span><span class="o">:</span><span class="mi">800001</span> <span class="n">mask</span><span class="o">:</span><span class="n">fce</span> <span class="n">ignored_mask</span><span class="o">:</span><span class="mi">0</span> <span class="n">fhandle</span><span class="o">-</span><span class="n">bytes</span><span class="o">:</span><span class="mi">8</span> <span class="n">fhandle</span><span class="o">-</span><span class="n">type</span><span class="o">:</span><span class="mi">1</span> <span class="n">f_handle</span><span class="o">:</span><span class="n">ba2308008ff93497</span>
<span class="n">inotify</span> <span class="n">wd</span><span class="o">:</span><span class="n">c</span> <span class="n">ino</span><span class="o">:</span><span class="mi">121276</span> <span class="n">sdev</span><span class="o">:</span><span class="mi">800001</span> <span class="n">mask</span><span class="o">:</span><span class="n">fce</span> <span class="n">ignored_mask</span><span class="o">:</span><span class="mi">0</span> <span class="n">fhandle</span><span class="o">-</span><span class="n">bytes</span><span class="o">:</span><span class="mi">8</span> <span class="n">fhandle</span><span class="o">-</span><span class="n">type</span><span class="o">:</span><span class="mi">1</span> <span class="n">f_handle</span><span class="o">:</span><span class="mi">7612120038</span><span class="n">dfa8a3</span>
<span class="o">...</span>
</code></pre></div>


<!-- /wp:code -->

<!-- wp:paragraph -->

<p><a href="https://unix.stackexchange.com/a/426001">StackExchange
网站的一篇回答</a>详细描述了这个原理，但给出的命令不太好用，于是后面有人提交了<a href="https://unix.stackexchange.com/a/502812">新的回答</a>，里面包含了一个更好用的命令：</p>
<!-- /wp:paragraph -->

<!-- wp:code -->

<div class="highlight"><pre><span></span><code><span class="n">find</span> <span class="o">/</span><span class="n">proc</span><span class="cm">/*/fd   </span>
<span class="cm">   -lname anon_inode:inotify   </span>
<span class="cm">   -printf &#39;%hinfo/%f\n&#39; 2&gt;/dev/null   </span>

<span class="cm">   | xargs grep -c &#39;^inotify&#39;    </span>
<span class="cm">   | sort -n -t: -k2 -r </span>
</code></pre></div>


<!-- /wp:code -->

<!-- wp:paragraph -->

<p>以及一个展示效果很好的<a href="https://github.com/fatso83/dotfiles/blob/master/utils/scripts/inotify-consumers">脚本</a>。</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->

<p>下载并运行这个脚本，可以看到当前用户（如果以 root
身份运行，则显示所用用户的进程，取决于用户可以查看的进程信息的范围）的进程使用的
inotify 监控数，并从高到低排列：</p>
<!-- /wp:paragraph -->

<!-- wp:code -->

<div class="highlight"><pre><span></span><code><span class="err">   INOTIFY</span>
<span class="err">   WATCHER</span>
<span class="err">    COUNT     PID     CMD</span>
<span class="err">----------------------------------------</span>
<span class="err">    4606    27023  /usr/share/code/code /usr/share/code/resources/app/extensions/node_modules/typescript/lib/tsserver.js --useInferredProje</span>
<span class="err">    3309    27054  /usr/share/code/code /usr/share/code/resources/app/extensions/node_modules/typescript/lib/typingsInstaller.js --globalTy</span>
<span class="err">     190    24015  /usr/share/code/code /usr/share/code/resources/app/out/bootstrap-fork --type=watcherService</span>
</code></pre></div>


<!-- /wp:code -->

<!-- wp:paragraph -->

<p>在不打开 webpack dev 的情况下，排名前三的都是 Visual Studio Code，但
Visual Studio Code 的文件监控只使用了190个监控，是 Visual Studio Code
进程中使用监控数量最少的一个，因此 files.watcherExclude 设置是有效的。</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->

<p>前两个都和 Visual Studio Code 的内置插件 TypeScript and JavaScript
Language Features
有关，这是非常重要的插件，如果把它关了（打开插件面板，选择 Show Built-in
Extensions 选项，在 FEATURES 组找到它并禁用），可以看到 JavaScript
类型提示没了，自动完成没了，IDE 变成了用处不大的编辑器。</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->

<p>但有一部分问题可以解决，排名第二的
typingsInstaller.js，其作用是在网上自动搜索安装 @types 库，为没有在 npm
包中提供类型信息的依赖库提供类型提示，在 settings 中勾选
typescript.disableAutomaticTypeAcquisition
可以关闭这个自动安装进程，节省下来 3000 多个 inotify 监控数。</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->

<p>最后看一下 tsserver.js 监控的那4000多个项目都是什么，在 fdinfo
文件里只列出了
inode，而且是16进制的，根据<a href="https://unix.stackexchange.com/a/35310">这篇回答</a>的启发，我写了一个
python 脚本，用 debugfs（因此只支持 ext 文件系统）找出这些 inode
对应的文件路径，在 main() 部分定义了脚本运行的参数，fdinfo 是进程的
fdinfo 文件，dev_code 是 fdinfo 文件中块设备的代码，dev_file
是该块设备文件对应的块设备文件：</p>
<!-- /wp:paragraph -->

<!-- wp:code -->

<div class="highlight"><pre><span></span><code><span class="kn">import</span> <span class="nn">subprocess</span>
<span class="kn">import</span> <span class="nn">sys</span>


<span class="k">def</span> <span class="nf">main</span><span class="p">():</span>
    <span class="n">fdinfo</span> <span class="o">=</span> <span class="s1">&#39;/proc/28621/fdinfo/21&#39;</span>
    <span class="n">dev_code</span> <span class="o">=</span> <span class="s1">&#39;800001&#39;</span>
    <span class="n">dev_file</span> <span class="o">=</span> <span class="s1">&#39;/dev/sda1&#39;</span>
    <span class="n">max_inode_queries</span> <span class="o">=</span> <span class="mi">500</span>

    <span class="k">with</span> <span class="nb">open</span><span class="p">(</span><span class="n">fdinfo</span><span class="p">,</span> <span class="s1">&#39;r&#39;</span><span class="p">)</span> <span class="k">as</span> <span class="n">f</span><span class="p">:</span>
        <span class="n">inodes</span> <span class="o">=</span> <span class="n">get_inodes</span><span class="p">(</span><span class="n">f</span><span class="p">,</span> <span class="n">dev_code</span><span class="p">)</span>

    <span class="n">result</span> <span class="o">=</span> <span class="p">{}</span>
    <span class="n">inode_index</span> <span class="o">=</span> <span class="mi">0</span>
    <span class="n">total</span> <span class="o">=</span> <span class="nb">len</span><span class="p">(</span><span class="n">inodes</span><span class="p">)</span>
    <span class="k">while</span> <span class="p">(</span><span class="n">inode_index</span> <span class="o">&lt;</span> <span class="n">total</span><span class="p">):</span>
        <span class="n">path_info</span> <span class="o">=</span> <span class="n">get_inode_path_map_ext4</span><span class="p">(</span><span class="n">dev_file</span><span class="p">,</span>
                <span class="n">inodes</span><span class="p">[</span><span class="n">inode_index</span><span class="p">:</span><span class="n">inode_index</span><span class="o">+</span><span class="n">max_inode_queries</span><span class="p">])</span>
        <span class="n">inode_index</span> <span class="o">+=</span> <span class="n">max_inode_queries</span>
        <span class="n">result</span><span class="o">.</span><span class="n">update</span><span class="p">(</span><span class="n">path_info</span><span class="p">)</span>

    <span class="n">sorted_paths</span> <span class="o">=</span> <span class="nb">sorted</span><span class="p">(</span><span class="n">result</span><span class="o">.</span><span class="n">values</span><span class="p">())</span>
    <span class="k">for</span> <span class="n">path</span> <span class="ow">in</span> <span class="n">sorted_paths</span><span class="p">:</span>
        <span class="n">sys</span><span class="o">.</span><span class="n">stdout</span><span class="o">.</span><span class="n">write</span><span class="p">(</span><span class="s1">&#39;{}</span><span class="se">\n</span><span class="s1">&#39;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">path</span><span class="p">))</span>


<span class="k">def</span> <span class="nf">get_inodes</span><span class="p">(</span><span class="n">f</span><span class="p">,</span> <span class="n">dev_code</span><span class="p">):</span>
    <span class="n">result</span> <span class="o">=</span> <span class="p">[]</span>
    <span class="n">inotify_string</span> <span class="o">=</span> <span class="s1">&#39;inotify&#39;</span>
    <span class="n">dev_string</span> <span class="o">=</span> <span class="s1">&#39;sdev:{}&#39;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">dev_code</span><span class="p">)</span>

    <span class="c1"># get output from command</span>
    <span class="k">for</span> <span class="n">line</span> <span class="ow">in</span> <span class="n">f</span><span class="p">:</span>
        <span class="k">if</span> <span class="n">line</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="n">inotify_string</span><span class="p">)</span> <span class="ow">and</span> <span class="n">dev_string</span> <span class="ow">in</span> <span class="n">line</span><span class="p">:</span>
            <span class="n">result</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">get_inode</span><span class="p">(</span><span class="n">line</span><span class="p">))</span>

    <span class="k">return</span> <span class="n">result</span>


<span class="k">def</span> <span class="nf">get_inode</span><span class="p">(</span><span class="n">line</span><span class="p">):</span>
    <span class="n">fields</span> <span class="o">=</span> <span class="n">line</span><span class="o">.</span><span class="n">split</span><span class="p">()</span>
    <span class="k">for</span> <span class="n">field</span> <span class="ow">in</span> <span class="n">fields</span><span class="p">:</span>
        <span class="k">if</span> <span class="n">field</span><span class="o">.</span><span class="n">startswith</span><span class="p">(</span><span class="s1">&#39;ino:&#39;</span><span class="p">):</span>
            <span class="k">return</span> <span class="nb">int</span><span class="p">(</span><span class="n">field</span><span class="p">[</span><span class="mi">4</span><span class="p">:],</span> <span class="mi">16</span><span class="p">)</span>


<span class="k">def</span> <span class="nf">get_inode_path_map_ext4</span><span class="p">(</span><span class="n">dev_file</span><span class="p">,</span> <span class="n">inodes</span><span class="p">):</span>
    <span class="n">request</span> <span class="o">=</span> <span class="s1">&#39;ncheck &#39;</span> <span class="o">+</span> <span class="s1">&#39; &#39;</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="nb">map</span><span class="p">(</span><span class="nb">str</span><span class="p">,</span> <span class="n">inodes</span><span class="p">))</span>
    <span class="n">command</span> <span class="o">=</span> <span class="p">[</span><span class="s1">&#39;sudo&#39;</span><span class="p">,</span> <span class="s1">&#39;debugfs&#39;</span><span class="p">,</span> <span class="s1">&#39;-R&#39;</span><span class="p">,</span> <span class="n">request</span><span class="p">,</span> <span class="n">dev_file</span><span class="p">]</span>
    <span class="n">command_output</span> <span class="o">=</span> <span class="n">subprocess</span><span class="o">.</span><span class="n">check_output</span><span class="p">(</span><span class="n">command</span><span class="p">)</span>

    <span class="n">result</span> <span class="o">=</span> <span class="p">{}</span>
    <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">line</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">command_output</span><span class="o">.</span><span class="n">splitlines</span><span class="p">()):</span>
        <span class="k">if</span> <span class="n">i</span> <span class="o">&gt;=</span> <span class="mi">1</span><span class="p">:</span>
            <span class="n">fields</span> <span class="o">=</span> <span class="n">line</span><span class="o">.</span><span class="n">split</span><span class="p">()</span>
            <span class="n">inode</span><span class="p">,</span> <span class="n">path</span> <span class="o">=</span> <span class="n">fields</span>
            <span class="n">result</span><span class="p">[</span><span class="n">inode</span><span class="p">]</span> <span class="o">=</span> <span class="n">path</span>

    <span class="k">return</span> <span class="n">result</span>


<span class="k">if</span> <span class="vm">__name__</span> <span class="o">==</span> <span class="s1">&#39;__main__&#39;</span><span class="p">:</span>
    <span class="n">main</span><span class="p">()</span>
</code></pre></div>


<!-- /wp:code -->

<!-- wp:paragraph -->

<p>运行该脚本显示 tsserver.js 监控了 Visual Studio Code 打开的项目的 src 和
node_modules 目录，以及 Visual Studio Code 本身的
/usr/share/code/resources/app/extensions/node_modules/ 目录。</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->

<p>也许没有两全的解决方案，因为 Visual Studio Code
需要掌握这些目录的变化，以获得对项目的 JavaScript
代码的全局掌握，没有办法在不消耗 inotify 监控资源的情况下，获得实时的
JavaScript 代码洞察带来的便利。</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->

<p>也许最更本的解决方案，如 <a href="https://code.visualstudio.com/docs/setup/linux#_visual-studio-code-is-unable-to-watch-for-file-changes-in-this-large-workspace-error-enospc">Visual Studio Code
文档</a>中所说，是增加监控文件数量的上限，在
/etc/sysctl.conf 中增加下列内容：</p>
<!-- /wp:paragraph -->

<!-- wp:code -->

<div class="highlight"><pre><span></span><code><span class="err">fs.inotify.max_user_watches=524288</span>
</code></pre></div>


<!-- /wp:code -->

<!-- wp:paragraph -->

<p>这在多用户共享的服务器上可能造成问题，但在单用户独享的开发主机上，最多消耗256M（32位）或者512M（64位）的内存。</p>
<!-- /wp:paragraph -->
    </div>
<nav class="nav sidebar">
    <div class="siteinfo">
        <div class="sitename"><a href="..">coinfaces</a></div>
        <div class="tagline">Another Side of Life</div>
    </div>

        <a href="../categories/">categories</a>
        <a href="..">&larr; Home</a>
</nav><footer class="footer">
    <span>
        &copy; jpli &middot;
        under <a rel="license" href="http://creativecommons.org/licenses/by-sa/4.0/">CC-BY-SA 4.0</a>
    </span>
    |
    <span>
        Powered by:
        <a href="http://blog.getpelican.com/">Pelican</a>
    </span>
</footer></body>
</html>